{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Falcon LLM Demo\n",
    "\n",
    "Reference:\n",
    "\n",
    "[HuggingFace Falcon](https://huggingface.co/tiiuae/falcon-40b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install transformers --upgrade\n",
    "!pip install torch --upgrade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b7d88cbea34a4f1b8d2bf73f71dbee63",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/9 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The model 'RWForCausalLM' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MvpForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM'].\n",
      "/home/rowel/anaconda3/envs/diffusion/lib/python3.10/site-packages/transformers/generation/utils.py:1259: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)\n",
      "  warnings.warn(\n",
      "Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result: Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\n",
      "Daniel: Hello, Girafatron!\n",
      "Girafatron: Oh, hi, Daniel. How are you?\n",
      "Daniel: Fine, thanks.\n",
      "Girafatron: *looks around* Do you see any other magnificent creatures like me?\n",
      "Daniel: *looks around* No...\n",
      "Girafatron: Exactly. All other animals are insignificant when compared to my glorious majesty.\n",
      "Daniel: Oh?\n",
      "Girafatron: *looks at Daniel and sniffs* You don't believe in the glory that is the giraffe?\n",
      "Daniel: Well, I do, it's just...\n",
      "Girafatron: Do not speak! You are unworthy to converse with me. I\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "import transformers\n",
    "import torch\n",
    "\n",
    "model = \"tiiuae/falcon-40b\"\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model)\n",
    "pipeline = transformers.pipeline(\n",
    "    \"text-generation\",\n",
    "    model=model,\n",
    "    tokenizer=tokenizer,\n",
    "    torch_dtype=torch.bfloat16,\n",
    "    trust_remote_code=True,\n",
    "    device_map=\"auto\",\n",
    ")\n",
    "sequences = pipeline(\n",
    "   \"Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\\nDaniel: Hello, Girafatron!\\nGirafatron:\",\n",
    "    max_length=200,\n",
    "    do_sample=True,\n",
    "    top_k=10,\n",
    "    num_return_sequences=1,\n",
    "    eos_token_id=tokenizer.eos_token_id,\n",
    ")\n",
    "for seq in sequences:\n",
    "    print(f\"Result: {seq['generated_text']}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result: I am wheeled robot. I can navigate on a flat floor. I can move forward and backward. I can turn around. Supposed I want to look for a red cup on the floor. What should I do? I can’t see red. I can’t see. What to do?\n",
      "If I am equipped with a camera, I can see. But it is not the end. I can also see the objects I see from a different viewpoint and from different direction. I can see the red cup on the floor from different angles, even from the inside of the cup. I can rotate the camera around, move it up and down, left and right, forward and backward. I can rotate the camera around and take a photograph of myself and of what I see with another camera. I can do all that with the same camera as long as it is equipped with an electronic device that rotates the camera. I may even take a photograph from another camera with the same camera by means\n"
     ]
    }
   ],
   "source": [
    "sequences = pipeline(\n",
    "   \"I am wheeled robot. I can navigate on a flat floor. I can move forward and backward. I can turn around. Supposed I want to look for a red cup on the floor. What should I do?\",\n",
    "    max_length=200,\n",
    "    do_sample=True,\n",
    "    top_k=10,\n",
    "    num_return_sequences=1,\n",
    "    eos_token_id=tokenizer.eos_token_id,\n",
    ")\n",
    "for seq in sequences:\n",
    "    print(f\"Result: {seq['generated_text']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result: Give a complete visual description of the rambutan fruit. Describe in terms of shape, color, and texture.\n",
      "The rambutan fruit is a red-orange in color, and has the size of a small apple. It is round, like a cherry but much larger, and has spiny protrusions, which are similar to a lychee. The flesh is translucent and is white, and it is sweet and juicy.\n",
      "The rambutan is a tropical fruit that is native to Southeast Asia. It has a thick, red-orange rind, which is covered in soft, spiny hairs. Inside, the flesh is white and translucent, with a sweet, juicy flavor reminiscent of lychee. The fruit is commonly eaten fresh, or used in desserts and jams.\n",
      "The rambutan fruit is a tropical fruit that is native to Southeast Asia. The fruit is round with a red-orange rind that is covered in soft, spiny hairs. The\n"
     ]
    }
   ],
   "source": [
    "sequences = pipeline(\n",
    "   \"Give a complete visual description of the rambutan fruit. Describe in terms of shape, color, and texture.\",\n",
    "    max_length=200,\n",
    "    do_sample=True,\n",
    "    top_k=10,\n",
    "    num_return_sequences=1,\n",
    "    eos_token_id=tokenizer.eos_token_id,\n",
    ")\n",
    "for seq in sequences:\n",
    "    print(f\"Result: {seq['generated_text']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result: Given a photo, what are the visible parts of a rambutan fruit? Describe the visible parts in terms of shapes, colors, texture, object parts, counts, and relationships.\n",
      "What is a rambutan?\n",
      "The rambutan is a Southeast Asian tropical fruit with soft spiny red, orange or yellow hair-like skin and a large seed. Its flesh is juicy and sweet with a mild, refreshing and unique taste.\n",
      "The flesh can be white, red or yellow. It can range in size from 1.8 - 2.5 cm, although some have been recorded as big as 6 cm.\n",
      "In some places, this fruit is eaten fresh; however, it is more commonly processed into jams, jellies, beverages, sherbets, ice creams etc and is commonly used as a filling in cakes.\n",
      "The fruit's name in Malay (Malaysian) literally means \"hairy\", which may also refer to\n"
     ]
    }
   ],
   "source": [
    "sequences = pipeline(\n",
    "   \"Given a photo, what are the visible parts of a rambutan fruit? Describe the visible parts in terms of shapes, colors, texture, object parts, counts, and relationships.\",\n",
    "    max_length=200,\n",
    "    do_sample=True,\n",
    "    top_k=10,\n",
    "    num_return_sequences=1,\n",
    "    eos_token_id=tokenizer.eos_token_id,\n",
    ")\n",
    "for seq in sequences:\n",
    "    print(f\"Result: {seq['generated_text']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result: What are useful features for distinguishing a rambutan fruit in a photo?\n",
      "1. The size, weight and shape of the rambutan fruit is variable according to the variety (see “Varieties”).\n",
      "2. The skin is covered with soft, short, brown or red or golden brown hairs.\n",
      "3. The rambutan seed is embedded in the flesh and has a hard, smooth, brown shell.\n",
      "4. Rambutan flesh is white and juicy.\n",
      "5. Rambutan flesh tastes sweet, juicy and soft.\n",
      "6. Rambutan has a small seed which is edible.\n",
      "7. Rambutan skin is soft and easily removed.\n"
     ]
    }
   ],
   "source": [
    "sequences = pipeline(\n",
    "   \"What are the useful features for distinguishing a rambutan fruit in a photo?\",\n",
    "    max_length=200,\n",
    "    do_sample=True,\n",
    "    top_k=10,\n",
    "    num_return_sequences=1,\n",
    "    eos_token_id=tokenizer.eos_token_id,\n",
    ")\n",
    "for seq in sequences:\n",
    "    print(f\"Result: {seq['generated_text']}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "diffusion",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
